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1.  Introduction 


This  project  was  motivated  by  time  our  team  spent  embedded  in  an  infantry  unit  in 
a  forest  in  Columbus,  Georgia.  The  Soldiers  requested  an  unmanned  aerial  vehicle 
(UAV)  that  they  could  pull  out  of  a  pack,  turn  on,  and  then  fly  through  the  trees, 
find  a  clearing,  exit  the  canopy,  surveil  the  area  while  transmitting  video  data  back 
to  the  ground  station,  reenter  the  canopy,  and  return  to  the  operator.  We  have  chosen 
to  address  the  forest  environment  in  which  such  a  UAV  would  operate.  Forests  are 
highly  complex  environments  -  in  addition  to  natural  variations  in  illumination  due 
to  passing  clouds,  the  trees  create  irregularly  placed,  occasionally  mobile,  beams  of 
light.  Furthermore,  we  are  constrained  to  a  theoretical,  small-scale,  easily  man- 
packable  UAV  with  an  extremely  limited  payload  that  must  move  at  operational 
tempo,  currently  defined  by  the  Defense  Advanced  Research  Projects  Agency 
(DARPA)  as  20  m/s.1  We  anticipate  this  platform  will  have  the  bare  minimum 
number  of  sensors  -  possibly  only  a  single  monocular,  grayscale  camera  -  and  we 
have  designed  this  system  to  be  robust  to  such  a  possibility. 

To  enable  stable  flight  on  pocket-sized,  highly  dynamic  unmanned  aerial  systems 
(UASs),  the  control  loop  requires  extremely  high  update  rates,  but  such  platforms 
have  only  a  minimal  payload  to  handle  the  computational  burdens.  As  system  size 
decreases,  high  fidelity  sensors,  such  as  LiDAR,  become  too  large  to  carry, 
requiring  a  shift  to  smaller  sensors  for  state  estimation.  Optical  systems,  which 
easily  scale  down,  typically  use  either  stereo  vision  or  optical  flow  to  generate  state 
estimates.  Both  methods  rely  on  the  objects  in  the  scene  maintaining  a  static 
representation  for  correlation,  which  causes  them  to  be  susceptible  to  dynamic 
lighting-induced  errors.  Even  large  systems  that  can  carry  LiDAR  and  substantial 
computation  often  couple  high  fidelity  systems  with  optical  systems  to  increase 
their  effective  range.2  While  there  are  many  optical  navigation  techniques  available, 
we  have  chosen  to  limit  the  scope  of  this  problem  to  optic  flow  due  to  its  low 
computational  burden  and  proven  real-time  usability. 

Traditionally,  optic  flow,  including  variants  of  Lukas  Kanade,3  Elementary  Motion 
Detectors  (EMDs),4  and  Horn  and  Schunck,5  has  been  tied  to  a  static  representation 
of  the  world.  All  of  these  algorithms  assume  that  color  representations  remain 
consistent  from  one  frame  to  the  next,  allowing  perceived  motion  to  be  tracked  by 
observing  pixel-wise  changes  between  the  2  images.5  This  assumption  becomes  a 
problem  when  lighting  conditions  change  unpredictably,  a  fairly  common 
occurrence  outdoors,  as  it  becomes  nearly  impossible  to  track  the  motion  of 
individual  pixels  that  have  now  lost  their  primary  signature.  As  lighting  regularly 
ranges  from  80  lux  indoors  to  1000  lux  outdoors  on  an  overcast  day  to  over  130,000 
lux  in  direct  sunlight,6  this  is  a  substantial  problem. 
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2.  Related  Work 


Before  we  address  similar  research,  we  would  like  to  mention  auto-exposure  and 
similar  in-system  camera  settings,  which  deal  with  varying  lighting  conditions. 
While  such  adjustments,  including  histogram  shifts  and  other  sorts  of  calibrations, 
may  produce  an  image  that  is  similar  and  comprehensible  to  the  human  eye,  the 
actual  pixel  values  (as  represented  from  0  to  255  in  a  grayscale  image)  often  differ 
beyond  what  is  recognizable  for  an  algorithm  making  a  correlation  from  frame  to 
frame. 

At  present,  while  there  have  been  numerous  explorations  into  illumination 
invariance,  there  remains  no  standard  method  to  improve  the  performance  of  optic 
flow  systems  in  dynamically  lit,  complex,  novel  environments.  Illumination 
mitigation  techniques  run  the  gamut  from  computationally  expensive 
postprocessing  using  a  variety  of  filters  and  masks  on  the  imagery  to  preprocessing 
to  alter  the  imagery  before  it  goes  into  the  optic  flow  algorithms.7-21  Some,  such  as 
work  in  highly  constrained  environments  like  highways,  assume  that  the  light  will 
shift  evenly  over  the  entire  image.7,8  As  our  system  is  intended  for  a  forested 
environment,  such  constraints  were  felt  to  be  unrealistic.  For  similar  reasons,  we 
move  past  methods  that  perform  object  identification  and  compare  the  colors  found 
to  those  of  a  template  to  estimate  the  lighting  changes  and  correct  the  image  as  a 
whole.9  1 1  We  do  not  wish  to  correct  the  image  -  simply  to  use  it  to  compare  against 
another.  Understanding  what  it  represents  is  not  within  the  current  scope  of  our 
investigation. 

Much  closer  to  our  problem  space  are  methods  that  look  to  buttress  optic  flow 
estimation  through  the  use  of  additional  sensing  modalities,  such  as  LiDAR,12 
intensity  modulated  (IM),13  or  temperature  scans.14  While  we  acknowledge  the 
utility  of  these  methods,  they  do  not  address  our  chosen  problem  of  unaided  optic 
flow.  A  small-scale  UAV  is  highly  constrained  by  its  payload  capacity  and 
processor,  and  such  methods  are  often  infeasible.  Zimmer  et  al.15  investigate  an 
implementation  of  optic  flow  in  the  hue-saturation-value  (HS  V)  color  space,  where 
they  found  the  hue  channel  is  invariant  under  a  variety  of  illumination  changes  and 
does  not  show  marked  reaction  to  shadows  or  specularities.  We  feel  this  is  a  very 
interesting  avenue  of  investigation,  but  it  was  not  used,  as  our  target  platform  uses 
grayscale  cameras  since  we  are  attempting  to  reduce  the  computational  burden,  and 
tripling  the  number  of  pixels  involved  would  not  further  that  goal.  We  would  also 
like  to  differentiate  this  technique  from  gradient-based  optic  flow16,17  -  our  method 
preprocesses  the  input  to  any  optic  flow  algorithm,  but  is  not  one  itself. 

The  techniques  most  similar  to  our  approach  look  at  preprocessing  the  input  prior 
to  putting  it  through  an  optic  flow  algorithm.  Christmas18  shows  the  use  of  a  spatial 
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filter  to  reduce  the  effects  of  temporal  aliasing  on  image  pairs  with  a  large 
discontinuity.  This  work  found  that  in  an  experiment  with  constant  illumination  and 
even  motion  pattern  it  was  possible  to  reduce  temporal  effects  using  a  low  pass 
filter.  However,  due  to  the  sizes  of  the  filters  investigated,  they  predicted  difficulty 
in  real-time  applications.  Lempitsky  et  al.19  removed  the  effects  of  shadows  by 
subtracting  from  the  first  image;  the  result  of  that  image  was  convolved  with  a 
Gaussian  kernel.  Beyond  the  fact  that  their  work  continues  in  the  red-green-blue 
(RGB)  color  space  and  is  finally  computed  with  bicubic  interpolation,  which  is  far 
more  computationally  complex  an  approach  than  we  will  be  able  to  use  for  our 
application,  the  issue  is  this  technique  ignores  higher-order  variations.  Sellent  et 
al.2  come  at  the  problem  from  a  completely  different  angle,  choosing  to  deal  with 
natural  variation  in  lighting  by  extending  exposure  time  and  using  the  camera  itself 
to  prefilter  sharp  variations  out  of  the  imagery.  While  this  approach  is  not  practical 
during  flight,  it  is  a  clever  approach  for  static  platforms.  In  Sharman  and  Brad,21 
images  are  filtered  prior  to  use  in  an  optic  flow  algorithm,  and  like  the  work 
presented  by  Sellent  et  al.,20  they  have  used  a  Gaussian  smoothing  filter.  Although 
that  paper  presents  a  filter  tuned  specifically  for  a  Lucas-Kanade  implementation 
at  each  pyramidal  level,  it  too  suffers  from  computational  complexity  limiting  its 
real-time  applicability. 

Our  investigations  have  found,  that  unlike  standard  electro-optic  (EO)  imagery,  the 
double  derivatives  of  those  same  images  remain  fairly  stable  in  dynamic  lighting 
conditions.  Image  derivatives  provide  the  rate  of  the  change  of  local  intensity  -  in 
other  words,  highlighting  the  edges  and  corners.  Whether  the  scene  is  brightly  or 
dimly  lit  will  not  change  the  textures  of  the  objects  in  the  scene,  and  barring 
extremes,  such  as  near  darkness  or  image  saturation,  things  like  tree  bark  look  more 
or  less  the  same  no  matter  how  they  are  lit.  However,  with  image  derivatives,  there 
is  an  overall  reduction  in  the  amount  of  information  available.  Fine  details 
disappear,  color  is  removed,  and  boundaries  that  lack  a  sharp  textural  difference 
can  merge.  While  our  approach  has  been  discounted  in  prior  work  due  to  this  loss 
of  information,12  we  find  that  there  is  sufficient  complexity  in  realistic  outdoor 
environments  to  maintain  a  high  enough  information  content  to  allow  navigation 
and  control  using  only  the  double  derivative  image.  Even  in  relatively  sparse 
environments  such  as  images  with  large  tracks  of  sky,  where  the  estimation 
uncertainty  grows,  there  is  a  sufficient  quantity  of  infonnation  that  our  method  does 
not  experience  the  catastrophic  failures  associated  with  other  optical  methods. 

3.  Methodology 

The  method  presented  here  is  intentionally  very  simple.  Small-scale  aerial 
platforms  don’t  have  the  capacity  for  complex,  real-time  calculations  while  flying 
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at  Army-desired  operational  tempos.1  There  are  3  steps  to  our  method:  1)  solve  for 
the  double  derivative,  2)  apply  the  optic  flow  algorithm  of  choice,  and  3)  isolate  the 
true  flow  using  a  mask,  as  shown  in  Fig.  1 . 


step  one 


Fig.  1  Flowchart  of  our  proposed  method,  showing  the  3  steps  employed 

Standard  derivative  calculations  were  used,  incorporating  both  the  x  and  y 
components.  While  several  different  filter  operators  were  tried  during  the  course  of 
the  project,  we  chose  [1,-1]  (or  its  transposed  pair,  for  y).  Numerous  other  standard 
filters  (Gaussian,  Laplacian,  etc.)  were  tried  and  produced  no  significant  increase 
in  performance.  Due  to  the  continued  focus  on  reducing  the  total  number  of 
computations  required,  a  smaller  filter  operator  was  preferred: 


where  f  —  is  the  value  of  the  pixel  at  the  (x,y)  position,  and  the  second  spatial 
derivative  of/is  the  gradient  of  the  image  (V/). 

Those  areas  of  the  image  without  any  texture  will  result  in  no  detected  motion  from 
the  optic  flow  algorithms.  To  avoid  biasing  the  state  estimation  and  still  allow  for 
the  possibility  that  there  may  not  have  been  any  ego  motion,  we  created  a  mask  to 
put  over  the  resultant  optic  flow  vector  field.  This  mask  is  based  on  the  information- 
poor  areas  of  the  filtered  image.  Through  experimentation,  we  found  that  masking 
all  areas  where  the  double  derivative  image  pixel  value  was  below  10  (on  a  scale 
of  0  to  255)  removed  the  majority  of  the  spurious  data. 

A  note  on  complexity.  Let  n  =  w  x  h  be  the  number  of  pixels  in  each  image  frame. 
We  assume  the  use  of  optic  flow  algorithms  with  input  consisting  of  sequential 
pairs  of  frames  in  a  time  sequence.  In  the  case  of  Lucas-Kanade,  the  time 
complexity  of  each  iteration  of  optical  flow  is  dominated  by  the  computation  of  the 
Hessian,  which  is  linear  in  n  but  quadratic  in  the  number  of  warp  parameters 
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m  (  ~  0(nm2))}2  In  the  case  of  iterative  global  methods  such  as  Hom-Schunck, 
analysis  of  the  computational  cost  has  additional  dependencies  including  whether 
the  algorithm  is  allowed  to  converge.  At  its  core,  the  Horn-Schunck  algorithm  is 
the  Jacobi  iterative  method  applied  to  the  interior  of  the  image23  and  requires 
computation  of  first-order  partial  derivatives  with  a  complexity  at  least  linear  in  n.24 
A  highly  optimized  implementation  may  precompute  certain  pixelwise  quantities 
prior  to  entering  the  iterative  phase;  even  so,  it  has  been  demonstrated  practically 
that  Horn-Schunck  requires,  on  average,  substantially  more  computations  per  pixel 
than  Lucas-Kanade.25  Our  technique  is  a  preprocessing  step  requiring  a  single 
convolution  over  each  image  to  be  taken  as  input  to  the  chosen  optic  flow  algorithm. 
The  results  of  the  derivative  computation  may  be  stored  for  later  use  at  a  cost  no 
greater  than  that  of  the  input  image.  Let  k  be  the  size  of  the  convolution  kernel 
(k  =  2  in  the  method  suggested  above).  We  assert  that  the  complexity  of  our  method 
is  linear  in  n  and  the  subsequent  masking  operation  requires  constant  time  per  pixel, 
resulting  in  an  overall  complexity  0(nk  +  cn).  We  posit  that  this  complexity  is 
generally  dwarfed  by  that  of  the  downstream  optical  flow  algorithm. 

The  implementation  of  Lucas-Kanade26  employed  for  these  experiments  used  3 
pyramids  and  3  iterations.  That  of  Hom-Schunck27  used  an  alpha  of  1  and 
maximum  iteration  of  100.  In  both  cases,  the  algorithms  were  received  pre-tuned 
and  were  not  altered. 

We  are  aware  that  this  method  is  not  robust  to  the  “features”  created  by  strong 
shadows.  However,  from  our  experience  embedded  with  infantry  units  in  realistic 
conditions  and  point  of  view  (POV)  video  data  gathered  during  those  experiments, 
strong  shadows  remain  relatively  static.  We  are  more  concerned  with  dynamic 
changes  such  as  the  passage  of  clouds  overhead,  which  will  affect  all  shadows  and 
lighting  conditions  equally.  However,  we  have  begun  investigation  into  texture 
patterns,  which  may  mitigate  the  shadow  problem. 

4.  Data 


The  images  used  in  the  following  experiments  were  collected  in  a  field  at  Ft. 
Benning,  Georgia,  during  a  live  exercise28  using  a  GoPro  Hero3  and  in  an  office 
using  a  Logitech  C920  Webcam.  Stage  lighting  was  used  to  create  controllable  and 
repeatable  dynamic  lighting  conditions  for  the  indoor  space.  The  outdoor  data 
collection  occurred  on  a  cloudless  and  extremely  sunny  day  and  the  lighting  is 
considered  to  be  static. 

For  ease  of  comparison,  both  data  sets  were  collected  with  a  static  camera.  To  create 
the  companion  “dynamic  motion”  sets,  the  “static  motion”  sets  were  subsampled 
and  only  a  portion  of  each  successive  frame  was  used.  The  location  of  this  region 
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within  the  base  image  was  altered  by  a  known  number  of  pixels,  which  became  the 
ground  truth  for  the  optic  flow  estimations.  To  ensure  the  data  were  comparable, 
the  static  sets  used  an  identically  sized  subsampled  area  from  the  original  images  - 
although  in  this  case,  the  location  of  the  window  did  not  alter  between  frames.  Each 
dataset  is  comprised  of  100  sequential  images,  selections  of  which  are  shown 
below. 

There  have  been  several  datasets  created  to  test  visual  state  estimation  under  a 
variety  of  circumstances,  most  notably  Kitti29  and  Sintel.30  Both  sets  include  a 
variety  of  lighting  conditions;  however,  we  did  not  find  known  illuminance  values 
associated  with  the  imagery.  As  this  is  preliminary  work  in  which  we  hoped  to 
explicitly  measure  the  relationship  between  changes  in  illumination  and  the  amount 
of  noise  added  to  an  optic  flow  estimate,  we  chose  to  create  our  own  dataset.  In 
future  iterations  of  this  work,  we  plan  to  use  established  datasets  to  allow  for 
rigorous  comparison  between  methods. 

5.  Experimental  Scenarios 

In  the  following  scenarios,  the  phrase  “algorithms”  refers  to  both  Horn-Schunck 
and  Lucas-Kanade  with  both  standard  and  prefiltered  inputs. 

5.1  Scenario  1:  Static  Camera,  Static  Lighting 

To  ensure  that  any  error  was  due  to  the  input,  rather  than  the  algorithms,  we  first 
tested  the  performance  of  the  optic  flow  methods  using  the  statically  lit  dataset  and 
a  static  camera  location.  With  no  motion  and  no  sensory  noise,  an  ideal  output 
would  show  a  cluster  of  dots  around  the  (0,0)  point. 

5.2  Scenario  2:  Static  Camera,  Dynamic  Lighting 

In  this  case,  we  wished  to  measure  how  the  algorithms  responded  to  changes  in 
illumination  only.  With  a  static  camera  and  static  scene,  any  resulting  “motion” 
must  be  due  to  the  illumination  shift. 

5.3  Scenario  3:  Dynamic  Camera,  Static  Lighting 

To  determine  how  accurately  each  algorithm  responded  to  actual  motion,  we  tested 
them  using  a  dynamic  camera  position  and  static  lighting  condition.  In  this  scenario, 
any  deviation  from  ground  truth  must  be  due  to  errors  in  state  estimation  of  the 
algorithms  themselves. 
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5.4  Scenario  4:  Dynamic  Camera,  Dynamic  Lighting 


The  final  scenario  investigates  a  mobile  camera  in  dynamic  lighting  conditions.  We 
feel  this  is  the  most  similar  to  true  field  operation  conditions  and  shows  how 
accurate  the  algorithms’  state  estimation  is.  In  this  case,  deviation  from  ground  truth 
is  due  to  a  combination  of  motion  and  lighting  conditions. 

6.  Results 


The  following  graphs  represent  the  results  of  processing  the  aforementioned 
lighting  conditions  with  and  without  the  prefiltering  step.  In  each  figure,  the  same 
camera  path  is  shown  in  both  static  and  dynamic  lighting  conditions.  By  showing 
how  the  algorithms  respond  to  their  expected  input  (statically  lit  images),  as  well 
as  how  they  respond  to  the  dynamic  lighting,  one  can  see  which  errors  are  due  to 
the  implementations  of  the  algorithms  and  which  are  due  to  the  input  data  format. 
Furthermore,  we  include  the  same  implementation  with  and  without  the  prefiltering 
step  to  show  directly  the  difference  it  makes.  The  left  column  of  each  figure 
displays  the  actual  plotted  position  of  each  state  estimate  as  well  as  the  true  position, 
while  the  right  column  shows  the  Euclidean  disparity  between  the  estimate  point 
and  ground  truth  at  each  position  in  the  run.  Deviation  is  measured  in  pixels. 

Figure  2  shows  the  results  of  static  (a,b)  and  dynamic  (c,d)  lighting  on  optic  flow 
using  data  collection  with  a  stationary  camera  to  demonstrate  the  error,  which  is 
added  to  state  estimation  through  variation  in  lighting  alone.  The  camera  is  static; 
any  perceived  “motion”  must  be  due  to  estimation  error.  As  may  be  seen  in  the  top 
row,  all  algorithms  perform  as  expected,  showing  subpixel  motion  when  presented 
with  no  motion  and  static  lighting.  However,  in  the  bottom  row,  one  may  see  the 
results  of  varied  lighting  conditions  and  that  the  filtered  implementations  are  far 
more  robust  to  its  effects. 
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Fig.  2  Measured  responses  for  a  static  camera  position  and  static  picture  with  dynamic 
changes  in  lighting,  where  a)  and  b)  show  the  position  estimation  and  deviation  from  ground 
truth  for  scenario  1,  and  c)  and  d)  show  the  results  of  scenario  2 


Figure  3  shows  the  results  of  optic  flow  in  both  static  (a,b)  and  dynamic  (c,d) 
lighting  conditions  as  captured  by  a  dynamic  camera.  In  this  case,  motion  may  be 
due  to  either  an  actual  change  or  lighting,  although  as  may  be  seen  by  comparing 
the  top  (static  lighting)  and  bottom  (dynamic  lighting)  rows,  all  algorithms  perform 
well  in  static  lighting,  while  the  prc filtered  versions  are  far  more  robust  in  the 
dynamic  conditions. 
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Fig.  3  Measured  responses  for  a  dynamic  camera  position  and  static  picture  with  dynamic 
changes  in  lighting,  where  a)  and  b)  show  the  position  estimation  and  deviation  from  ground 
truth  for  scenario  3,  and  c)  and  d)  show  the  results  of  scenario  4 

As  anticipated,  the  standard-input  optic  flow  algorithms  produce  nearly  perfect 
results  in  static  lighting  conditions,  with  a  mean  error  of  1.6  pixels  for  both 
algorithms  and  a  standard  deviation  of  0.77  and  0.79  for  Hom-Schunck  and  Lucas- 
Kanade,  respectively.  The  prefiltered  version  results  overshot  the  mark  slightly, 
with  mean  errors  of  4.8  and  5.9  pixels  and  standard  deviations  of  2.8  and  3.5  pixels 
for  Hom-Schunck  and  Lucas-Kanade,  respectively. 

However,  in  dynamic  lighting  conditions,  the  prefiltered  results  remain  very  close 
to  the  ground  truth  and  do  not  deviate  significantly  over  time.  In  this  case,  they 
averaged  1.8  and  4.1  pixels  of  error  with  standard  deviations  of  0.7  and  2.2  pixels 
for  Hom-Schunck  and  Lucas-Kanade,  respectively.  Conversely,  the  standard  input 
algorithms  are  not  reliable  under  such  conditions,  with  mean  errors  of  10.9  and 
3 1 .45  pixels,  with  standard  deviations  of  5. 1  and  33.7  pixels  for  Horn-Schunck  and 
Lucas-Kanade,  respectively.  The  prefiltered  optic  flow  does  better  in  dynamic 
lighting  than  it  does  in  static,  and  we  are  currently  investigating  how  this  can  be 
generally  applied. 
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To  begin  testing  the  boundaries  of  this  method,  we  created  the  experiment  presented 
in  Fig.  4  by  comparing  a  reference  image  at  a  known  illumination  to  a  series  of 
other  images  of  the  same  object,  captured  at  a  variety  of  known  illumination  levels. 
The  algorithms  used  here  are  identical  to  those  used  earlier  in  the  report.  For 
context,  a  change  of  100  lux  is  within  the  deviation  allowed  within  a  well-lit  office. 
Realistically,  changes  in  illumination  are  often  far  more  radical.  A  cloud  passing  in 
front  of  the  sun  can  easily  case  variations  on  the  order  of  5,000  lux.  While  the 
degree  of  variation  in  illuminance  from  frame  to  frame  is  dependent  on  the  frame 
rate  of  the  camera  and  the  conditions  the  system  is  operating  in,  our  method’s 
robustness  to  changes  of  over  600  lux  per  transition  and  our  planned  frame  rate  of 
30  fps  gives  us  confidence  that  even  and  indoor/outdoor  transition  of  10,000  lux 
would  be  manageable. 


Noise  Due  to  Lighting 


Lucas-Kanade 
Prefiltered  LK 


>  »  4****  Mt  +  i _ ±_ 


0  100  200  300  400  500  600  700  800  900 

Delta  lux 


Fig.  4  Filtered  and  unfiltered  optic  flow’s  responses  to  extreme  variations  in  illuminance 

7.  Conclusion 


The  primary  assumption  of  traditional  optic  flow  is  that  illumination  remains 
constant,  and  under  such  conditions,  it  perfonns  admirably.  However,  the  real 
world  is  a  dynamic  and  irregular  place  and  any  system  that  intends  to  operate  within 
it  must  be  robust  to  constantly  changing  conditions.  As  shown  in  Figs.  2  through  4, 
simply  by  changing  the  input  from  a  standard  image  to  the  double  derivative  of  that 
image,  we  are  able  to  produce  a  significant  increase  in  position  estimation  accuracy 
under  dynamic  lighting  conditions.  The  next  big  push  in  robotics  is  for  small, 
autonomous  systems  to  quickly  navigate  complex  environments,  specifically, 
below  the  canopy  in  forests.1 

We  have  found  that  prefiltering  is  a  useful  tool  for  real-world  navigation  and 
particularly  for  size,  weight,  and  power  (SWaP)-constrained  systems  that  cannot 
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carry  more  than  a  single  camera.  It  is  computationally  reasonable,  as  creating  an 
image  double  derivate  is  substantially  less  expensive  than  the  host  optical  flow 
algorithm  and  does  not  require  any  sort  of  a  priori  infonnation.  Our  approach  is 
robust  to  significant  variations  in  lighting,  while  also  being  very  easily 
implemented  and  added  to  existing  system.  Despite  the  information  lost  by  taking 
the  derivative  of  the  image  prior  to  optic  flow  calculations  and  the  subsequent 
masking  of  the  dead  zones,  we  find  that  sufficient,  relevant  information  persists  for 
this  method,  producing  useful  results. 

8.  Future  Work 


There  is  still  a  great  deal  of  work  to  be  done  in  order  to  make  optic  flow  robust  to 
the  vagaries  of  outdoor  operation.  We  plan  to  investigate  how  best  to  identify  and 
eliminate  features  due  to  strong  shadows  rather  than  actual  environmental  features. 
Despite  the  strong  edges  and  image  variations  caused  by  shadows,  the  textures  of 
the  objects  within  the  scene  remain  the  same.  We  have  considered  a  method  that 
compares  the  patterns  and  significant  deviations. 

Another  typical  source  of  error  are  dynamic  backgrounds,  such  as  windblown  trees 
and  grass.  Such  objects  have  fixed  locations,  but  nonetheless  add  a  great  deal  of 
noise  to  the  vector  field  and  disrupt  the  flow  patterns  usually  used  to  determine  ego 
motion  and  the  presence  of  outside  agents.  Furthermore,  when  resident  on  a  vehicle, 
there  are  likely  to  be  ego-motion  errors,  such  as  vehicle  drift  and  vibration,  which 
will  need  to  be  accounted  for. 

We  look  forward  to  combining  all  of  these  problems  in  future  investigations  into 
dynamic  environments  with  highly  constrained  systems. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


DARPA 

Defense  Advanced  Research  Projects  Agency 

EMD 

Elementary  Motion  Detector 

EO 

electro-optic 

HSV 

hue-saturation-value 

IM 

intensity  modulated 

POV 

point  of  view 

RGB 

red-green-blue 

SWaP 

size,  weight,  and  power 

UAS 

unmanned  aerial  system 

UAV 

unmanned  aerial  vehicle 
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